(19) 



J 



CuropJIlsche* Patentamt 
European Patent Office 
Office europeen des breveta 



(12) 



(43) Date of publication: 

05.03.2003 Bullet In 2003/10 

(21) Application number: 01 830562.3 

(22) Date of filing: 31 .08.2001 



(ii) EP1 289 309 A1 

EUROPEAN PATENT APPLICATION 

(51) mtci7: H04N 9/04, G06T5/20 



(84) Designated Contracting States: 


(72) 


inventors: 


AT BE CH CY DE DKES R FR QB QR IE ITU LU 


• 


Bosco, Angelo 


MC NLPTSETR 




95014 Glarre (Catania) (IT) 


Designated Extension States: 


• 


Mancuso, Massimo 


ALLTLVMKRO SI 




20052 Monza (Milano) (fT) 


(71) Applicant: STMIcroelectronlce S.r.l. 


(74) 


Representative: Bottl, Mario . 


20041 Agrate Brlanza (Milano) (IT) 




Botti & Ferrari S.r.l., 






Via Locatelll, 5 






20124 Milano (IT) 



(54) Noise filter for Bayer pattern image data 

(57) A method of filtering and an fcnage filter (10) is 
disclosed. The filter is provided for a digital camera in- 
cluding Image sensors sensitive to light, a color filter 
placed over sensitive elements of the sensors and pat- 
terned according to a Bayer mosaic pattern layout and 
an Interpolation algorithm Joining together the digital In- 



formation provided by differently colored adjacent pixels 
in said Bayer pattern. The filter (10) is adaptive and in- 
cludes a noise level computation block (26) for operating 
directly on a said Bayer pattern data set of for each color 
channel thus removing noise while simultaneously pre- 
serving picture detail. 
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Description 

Field of Application 

s [0001 J This Invention relates to a nobe filter, and, more particularly to a noise filter for Bayer pattern fcnage data. 
[0002] The Invention also relates to a method of filtering noise in a Bayem pattern Image data. 

Prior Art 

w [0003] Digital cameras generate a dataf lie that represents an image acquired by the camera. Generally, the camera 
acquires the Information from the light/color sensors In the camera In a CFA (camera filter array) format A popular 
format for the CFA is a Bayer mosaic pattern layout, shown In Figure t . In a Bayer pattern, each pixel contains Infor- 
mation that are relative to only one color component, for instance, Red, Green or Blue. Typically, the Bayer pattern 
Includes a green Information In every other space, and, In each row, either a blue or a red pixel occupies the remaining 

15 space. For Instance, as seen In Figure 1, row one alternates between green and red pixels, and row two alternates 
between green and blue. The end result Is a mosaic made of red, green and blue points, where there are twice as 
many green points as red or blue. This can accurately represent an image because the human eye is more sensitive 
to the green data than either the red or blue. 

[0004] A typical camera includes a charge coupled device (CCDs) or CMOS image sensor that is sensitive to light. 
so These image sensors are sensitive only to the Intensity of light falling on it and not to the light's frequency. Thus, an 
Image sensor of this kind has no capability to differentiate the color of light falling on. It. 

[0005] To obtain a color image from a typical camera sensor, a color filter is placed over the sensitive elements of 
the sensor. The filter can be patterned to be like the Bayer pattern discussed above. Then, the Individual sensors are 
only receptive to a particular color of light, red, blue or green. The final color picture Is obtained by using a color 

& interpolation algorithm that joins together the information provided by the differently colored adjacent pixels. 

[0006] The Images produced by digital still cameras, especially ones produced by CMOS technology, suffer from 
noise that Is Inherently present In the sensor when the Image Is captured. Thus, some modification of the datafile 
produced by the Image sensor is needed. Oftentimes this modification comes as a filter or algorithm run on the image 
sensor data (the Bayer Pattern) to create a more realistic Image. Processes are performed on It In an Imaging process 

30 chain, e.g. white balance, gamma correction, etc, as shown In Figure 2. Finally, the corrected Image Is compressed 
and stored in memory as a color image. 

[0007] One problem with current Image processing In current digital camera systems is that noise artifacts are difficult 
to differentiate from detail in the image Itself. If noise artifacts are thought to be detail and left unfiltered, or details are 
thought to be artifacts and are filtered, image degradation occurs. 
55 [0008] Another problem exists in that current filtering methods operate on the Image data as a whole and do not 
differentiate between the individual color channels making up the image. 

[0009] The technical problem solved by this invention is one of how to best filter a complex image file for noise, while 
at the same time preserving detail In the Image. 

40 Summary of the Invention 

[001 0] The solution to the above-stated problem is that of using an adaptive filtering system In a Bayer pattern Image 
independently for each of the color channels. The adaptive filter allows for a differing amount of filtering action performed 
based on the particular area of image and color of the currant pixel. For example, the filtering action could be milder 
in textured areas of a color channel, where the inconsistencies In data could be related to detail in the scene, but 
stronger In uniform areas of the color channel where the aberration Is more likely to be noise. 
[001 1 ] Based on this principle, the technical problem is solved by an image filter as defined in Claim 1 , and further 
by a method of filtering noise as defined In Claim 6. 

[0012] The features and advantages of the method and device according to the invention will be apparent from the 
so following description of an embodiment thereof, given by way of non- limitative example with reference to the accom- 
panying drawings. 

Brief description of the drawings 

55 [0013] 

Figure 1 is a diagram of a Bayer pattern mosaic layout used in digital cameras of the prior art. 
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Figure 2 Is an example functional block diagram of Image processes of a prior art digital camera. 

Figures 3A, 3B and 3C are diagrams of filter masks for various colors In the Bayer pattern data, according to an 
embodiment of the Invention . 

Figure 4 Is a flow diagram Illustrating example processes that are present In embodiments of the Inventive adaptive 
noise filter. 

Figure 5 Is an example block diagram showing Interrelationships of processes In the adaptive noise filter according 
to an embodiment of the Invention. 

Figure 6A shows an example working window of a set of green pixels currently being filtered. 

Figure 6B shows an example working window of a set of non-green pixels currently being filtered. 

Figure 7 Is a chart of a human vision threshold factor for nearby pixels as It relates to the brightness of the desired 
pixel. 

Figure 8A is a graph of the K,, parameter as a function of the maximum distance value Dmax when the current 
pixel Is green. 

Figure SB is a graph of the K,, parameter as a function of the maximum distance value Dmax when the current 
pixel Is not green. 

Figure 9 is a graph showing a different way to generate the K„ parameter using a gain variable. 

Figure 1 0 Is an example diagram of a fuzzy function able to fuzzily relations of pixels according to an embodiment 
of the invention. 

Figure 11 is a diagram showing an edge of a Bayer pattern dataset. 

Figure 12 is a diagram of an Image formed by pixels, highlighting the border made of two pixels. 
Figure 13A is an original control image. 

Figure 13B is the image of Figure 13A after It has been modified by an adaptive noise filter according to an em- 
bodiment of the invention. 

Figure 1 4 Is a diagram showing the strength levels of Altering as used In the conversion of Figure 1 3A to Figure 1 3B. 
Figure 15 Is an example test Image having an Increasing amount of noise from top to bottom. 
Figure 16 is a graph of noise power computed for different noise levels with reference to Figure 15. 
Detailed Description 

[0014] Embodiments of the invention operate directly on a noisy Bayer pattern data set to adaptively remove noise 
while simultaneously preserving picture detail. Once filtered, the Bayer pattern data Is sent to the other standard Image 
processing chain processes to produce the image stored on a memory medium. 

[0015] Some of the calculation steps are similar to those found in US patent 6,108,455, which is assigned to the 
applicant, and Incorporated herein In Its entirety by reference. 

[0018] In one process, an algorithm filters the image acquired by the sensor of a digital still camera. Thus, the input 
Image Is In the Bayer pattern format. This noise filter computes a replacement value for the pixel under processing In 
the processing window based on a weighted average of the absolute values of degrees of similarity between the target 
pixel and the plurality of neighboring pixels. The noise filter recursively processes the noise levels to adapt its strength 
depending on the local features of the acquired image , as well as to adapt to changing noise levels of neighboring pixels. 
[001 7] Two different filter masks are used to filter the three color channels of Bayer pattern data, red, green and blue. 
One mask operates on the green channel exclusively. The other mask operates on both the red and blue channels, 
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but not simultaneously. 

[0018] Three different noise level estimations are computed; one for each color channel of the image, red, green 
and blue. The filter can be Instructed to change its strength separately for each of the red, green and blue channels. 
Generally, the red and blue pixels making up the red and blue channels can be filtered at a higher strength than the 
5 green pixels, due to the way the human eye perceives color, but the process Is capable of adjusting to any strength In 
any color channel. 

[0019] As stated above, two filter masks are used, depending if the green channel, or either the red or blue channels 
are being filtered. The different filter masks are illustrated In Figures 3A, 38 and 3C. Figure 3A shows a filter mask for 
the green pixels In the Bayer pattern, according to an embodiment of the Invention. Generally, for a particular green 

10 pixel being filtered, the signal levels of the pixel being filtered and Its neighboring pixels are examined. For Instance, 
with reference to Figure 3A, a green pixel GO is the target pixel; a working window Including a portion of the pixels 
neighboring the target pixel GO Is established, for Instance by Including the neighboring pixels G1, G2, G3, G4, G5 r 
G6, G7, and G8. Similarly, with reference to Figure 3B, a working window is established with a filter mask for the Red 
channel. This filter mask differs from the filter mask shown In Figure 3A because, as discussed with reference to Figure 

*5 1, the pixel arrangement in the Bayer pattern is different for the green channel than either the red or blue channels. In 
the red filter mask, shown In Figure 3B, a target pixel R0 Is surrounded by Its neighboring pixels R1 , R2, R3, R4, R5, 
R6, R7, and R8. Similarly, the blue filter mask, shown in Figure 3C, has a target pixel BO surrounded by its neighboring 
pixels B1 , B2, B3, B4 t B5, B6, B7, and BB. In each of the cases for red, green and blue, there Is one target pixel and 
eight neighboring pixels, which will be referred to generically as TO and T1-T8, respectively. 

*0 [0020] The input image in the Bayer pattern format Is scanned sequentially row by row, beginning on the top row, 
from left to right. In embodiments of the Invention, the processing windows for the red and blue channels are Identical, 
while the processing window for the green channel is different. When the picture Is processed, only one of the two 
types of processing windows Is selected, and thus, the selection Is based dependent on whether the current pixel color 
Is green. 

29 [0021 ] Figure 4 is a flow diagram illustrating example processes that are present in embodiments of the inventive 
adaptive noise filter. The adaptive noise filter 1 0 begins with a step of copying Image borders 1 2, which will be discussed 
below. As described above, a determiner 16 then examines the current pixel TO to see If It Is green or not. If TO Is a 
green pixel, a green working window 1 8, such as seen In Figure 3A Is produced. Instead, if TO is not green, a red/blue 
working window 20 Is produced such as seen In Figures 3B or 3C. 

so [0022] Once the mask and filter window Is selected, a distance calculation Is made In a difference calculator 22. In 
the difference calculator 22, an absolute value difference Di is determined between the target pixel TO and its neigh- 
boring pixels. Each difference Dl represents the difference in between the value of the target pixel TO and the values 
of Its neighboring pixels. The difference calculator 22 computes the absolute values of the differences Di between the 
target pixel TO value of the current working window and the values of its neighboring pixels T1-T8 to produce the 

35 difference values D1 , D2, D3, D4, DS, DS, D7, and D8. 

[0023] Once the differences In pixel values are calculated, the results are sent to a maximum and minimum block 
24. In that block, the maximum and minimum distance values are computed from the list of difference values DI -D8. 
The output of the max/mln block 24 Is the maximum difference Dmax and the minimum difference Dmln between the 
target pixel TO and Its neighboring pixels T1 -T8 of the current working window 1 8 or 20. 

40 [0024] These minimum and maximum differences are passed to a noise level computation block 26 , which, without 
using autoexposured data, makes an estimation of a noise level NL associated with the processing window 18 or 20. 
Depending on whether the processing window 18 (green) or processing window 20 (red or blue) Is the current window, 
different NL calculations are made and stored simultaneously, one for each color channel. Depending on the color of 
the current pixel TO, the noise level block 26 updates the relative noise level estimation for that particular color. Thus 

45 the noise level block 26 estimates the noise level according to the following calculations: 

NL R (t-1) = KJM) * D max (M) + [1 - K n (t-1)] * NL R (t-2) (1) 
NL 0 (t-1)= K^t-1) • D max (t-1) + [1 - K n (t-1 )] * NL Q (t-2) (2) 



NLe (t-1) = K n (t-1) * D max (M) + [1 K n (t-1)] * NL 9 (t-2) (3) 

55 

[0025] where NL, Is the noise level of the particular noise channel, x; Kn Is a parameter that determines the strength 
of filtering to be performed by the adaptive noise filter 10, as discussed In the above-referenced US patent and as 
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further discussed below, and wherein D max Is the maximum distance value computed for the particular color channel 
In max/ml n block 24. 

[0026] Examples of the Interrelationship of different processes can be seen In the example block diagram of the 
Inventive adaptive noise filter 1 0 In Figure 5. In that figure the difference calculator 22 takes Its Input from the current 
s processing window 1 8 or 20. The output from the difference calculator 22 Is passed to the max/mln block 24, which 
passes Dmax and Dmln values for each color to the noise level block 26. The Kp parameter Is also an Input to the 
noise level block 26. 

[00271 Generally, these processes used In the adaptive noise filter 1 0 can be Implemented In software or hardware, 
although hardware Is generalry preferred for performance reasons. It Is fundamental to process the Input noisy image 
10 as quickly as possible using a minimum amount of resources to perform all of the computations. In order to achieve 
these goals, the noise level computation block 26, according to an embodiment of the Invention, uses a recursive 
Implementation. Typically, two image buffers can be used to execute the entire filtering process. One buffer (an Input 
buffer) contains the image to be filtered; the other buffer (the output buffer) contains the filtered image. 
[0028] In order to make best use of the resources required by the processes, the filtered value of the current target 
1$ pixel TO is computed using the pixel values that have already been filtered. For example, if TO is the target pixel, and 
T1 , T2, T3 and T4 have already been processed, then T5, T6, T7 and T8 have yet to be processed. Keeping track of 
already filtered pixels Is a good practice to reduce the computational requirements of the various processes. As the 
Bayer pattern Image Is scanned.row by row, the already filtered values are available In memory and they can be used 
to filter the next pixel without having to reload into the memory the old unfiltered values of T1 , T2, T3 and T4. This 

& process can be used for both the Green working window 1 8, and the Red/Blue working window 20. 

[0029] Figure 6A shows an example working window 1 8 of a set of Green pixels, GO (the target pixel), G1 , G2, G3 
and G4, which have already been filtered, and G5, G6, G7 and G8 which have yet to be filtered. 
Figure 6B shows an example working window 20, which In this case happens to be blue, although It would be Identical 
for the red color channel. In Figure 6B, BO Is the target pixel being filtered, B1 , B2, B3 and B4 have already been 

25 filtered, and B5, B6, B7. and B8 have not been filtered yet. 

[0030] Returning back to Figure 5, an human visual system (HVS) evaluator 30 Is shown, which receives Its Input 
from the working windows 1 8 and 20. The HVS evaluator 30 factors the human eye's response to a brightness change 
according to the function shown In Figure 7. In that figure, the X-axis measures possible brightness values on a scale 
between 0 and 1 023, while the Y axis charts the threshold level where the human eye can determine a difference In 

30 grey level values between a current pixel and Its neighboring pixels. As shown In Figure 7, when a pixel Is In the middle 
(x=512) of the dark <x=0) and the brightest value (x=1 023), the HVS threshold of the human eye is the lowest. Therefore, 
the HVS threshold of the human eye Is the highest when the pixels are very bright or very dark, as shown In Figure 7. 
[0031 ] The factor from the HVS evaluator 30 is summed with the output of the noise level block 26 and provide to a 
local feature inspector block 32 (Figure 5). That block 32 computes the K„ parameter that is used to modulate the 

35 strength of the filtering process to be executed on the current pixel TO in the noise level block 26. Depending on the 
color of thecurrent pixel TO, either green or red/blue, two different local feature inspector functions are used. Specif Icaily, 
If the current pixel being filtered is green, then the standard local feature inspector block is used. This is illustrated in 
Figure 8A, which Is a graph of the K„ parameter as a function of ths maximum distance value Dmax when the current 
pixel Is green. Shown In Figure 8A Is a value Th 3 , which Is the noise value plus the HVS value for the current pixel TO, 

4° (Th 3 = NL + HVS). In Figure 8A, the K„ parameter Is a straight line starting at 1 (when Dmax Is at Its minimum), and 
extending down to Th 3 . Figure 8B, conversely, Is a graph of the K„ parameter as a function of the maximum distance 
value Dmax when the current pixel Is not green. Figure 8B Introduces another value to the calculation of the K„ pa- 
rameter, which is the noise level NL. Tnus, when Dmax has a value that is less than or equal to the NL, then the K n 
parameter will be 1 , otherwise the K„ parameter has a value found along the line between 1 and Th^, This augments 

45 the filter strength on the red and blue channels compared to the green channel, because the K n parameter for the 
green channel can only be 1 when Dmax Is at 0, while the K n parameter for the red and blue channels Is always 1 
when Dmax Is less than or equal to the NL. 

[0032] An additional way to control the degree of filtering the Bayer pattern data la to Include a gain factor used by 
the autoexposure control (AEC). Such block including AEC is Ulustrated as 34 in Figure 5. The degree of filtering can 

so be modulated depending on the current gain factor used by the AEC block 34. The autoexposure process in a digital 
camera generates an appropriate set of exposure settings and gain settings In order to obtain a correctly exposed 
scene. If the Image scene is correctly exposed, then no gain factor needs to be applied to the Image. If, on the other 
hand, the Image cannot be correctly exposed, then the autoexeposure process of the camera computes a gain factor, 
which is a sort of coefficient that Is applied to the image. The higher the gain, the brighter the Image. There are some 

55 drawbacks to this universal system, however If the gain is set to a too high value, noise is also enhanced in the image. 
Therefore, when the gain is high, the noise filter must act with a higher strength, and when the gain is low, or at zero, 
then the noise filter should act with a lower strength. 

[0033] The adaptive noise filter 10 can Include a process that uses the following approach, as Illustrated with refer- 
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encs to Figure 9. That figure shows how an extra factor (the AEC gain factor) can be used to create the K n parameter. 
If G Is the current AEC gain factor computed by the AEC block 34 of Figure 5, and Gmax Is the Maximum value of the 
gain factor, which will depend from the Image sensor), then a ratio of G/Gmax can be calculated. This ratio, as shown 
In Figure 9, Is multiplied by the noise level factor NL to detemnlne the highest Dmax value that will have the K„ parameter 
5 equal to 1 . This differs to the graph shown In Figure 8B, which did not Include the gain G/Gmax ratio factor. Thus, If to 
obtain the best exposed Image G Is set to Gmax, then this means that the G/Gmax ratio Is 1 , so the threshold that 
separates the maximum degree of filtering (K„ = 1 ) from the other degrees of filtering Is the noise level NL. Conversely. 
If the correctly exopsed picture Is reached with a gain G such that 0<= G <= Gmax, then NL*G/Gmax <= NL, because 
G/Gmax <=1 , So, the treshold that Is used to discriminate between the higher degree of filtering from the other degrees 

io is lower, hence the filter strength is lowered. This technique can be applied to all of the three color channels, or It may 
be usedon onlythe red/blue channels, while the green channel may be filtered In the standard mode shown In Figure 8A. 
[0034] With reference to Figure 6, It may be necessary for the AEC block 34 to provide Its output prior to the time 
the noise levels are calculated by the noise level computation block 26. This is not problematic because the noise level 
computation block 26 and other portions of the adaptive noise filter 10 are not restricted to be the first element of an 

15 image processing chain. The adaptive noise filter 1 0 can actually be positioned much later In the image manipulation 
process (Figure 2), such as within the defect correction block, If necessary. When the adaptive noise filter 10 Is posi- 
tioned In the defect correction block of the image processing chain, it accepts as an input the noisy pixel. Then, the 
de-nolsed value Is passed as output from the adaptive n olse filter to the defect correction algorithms In order to generate 
the final den oised and defect corrected pixel of the Bayer pattern to be passed to the rest of the image processing chain. 

^0 [0035] The final process to be performed on the pixel value is based on fuzzy logic. The membership computation 
block is associated to a fuzzy function bounded by two threshold values Th1 and Th2. The fuzzy function fuzzltles the 
relation "the target pixel TO and its neighbor pixel Tl are similar", where 71 is any of the neighboring pixels T1 - T8. For 
each neighboring pixel Tl of TO, a coefficient Kl Is computed, which weights the similarity between T and Tl. These 
weight coefficients Ki are later used in the final determination of the filtered value. Thus, for every pixel TO, there will 

2s be eight coefficient values, K1-K8, where K1 weighs the similarity between TO and T1, K2 weighs the similarity between 
TO and T2, etc. , up to K8. 

[0036] An example diagram of a fuzzy function able to fuzzily the above relation Is shown In Figure 10. The two 
threshold values Th1 and Th2 can be factored according to the following equations: 

™1 - Kn« * D max » + P ' *„ R] * D mh (t) (4) 



Th2 = K n (t) • D raax (t) + [1- K n (t)J • D mln (t) * «D max (t) + D mln (t)] 12) (5) 

35 

[0037] The value of Ki can be expressed through the following relations: 

KU1,whenDi<=Th1 (6) 

40 

KJ = (Th2 - Dl) / (Th2 - Th1), when Th1 < Dl < Th2 (7) 



45 Ki = 0, when Th2 <= Di (B) 

[0038] These values are calculated in a threshold calculation block 36 shown in Figure 5, which accepts as inputs 
the threshold factor Kn, Dmln, and Dmax, and produces the threshold values Th1 and Th2 shown above. 
[0039] These threshold values are Input to a coefficient calculator 38, which calculates the KJ values of equations 6, 
50 7and8. 

[0040] A final output of the adaptive noise filter 10 Is the filtered value of the current target pixel TO. By repeating 
this process sequentially for every pixel produced by the camera sensor, the final filtered Image is obtained at Its output. 
The output of the noise filter 1 0 can be shown to be as follows: 

55 
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(9) 



30 



35 



40 



45 



[0041 ] As stated previously, the adaptive noise filter 1 0 works on Bayer pattern source data. Considering the shape 
of the processing windows it can be noticed that it is not possible to process those pixels belonging to the first and last 
two rows of the Image, and those pixels belonging to the first and last two columns, because there are not enough 
pixels to create a full working window 1 6 or 20 (Figures 3A, 3B and 3C). mis concept is Illustrated with reference to 
Figure 11, a Bayer pattern showing the first five columns and five rows of an edge of an Image is shown. It Is Impossible 
to create a full working window 1 8 for the first green pixel located In the first column and first row, because it does not 
have enough surrounding pixels. 

[0042] Therefore, as illustrated in Figure 12, some embodiments of the invention leave the two-pixel-wide borders 
of the Image In their original condition, and do not filter these particular borders. That Is why, with reference to Figure 
4, the border copying step 12 Is the first step to be performed. Thus, In order to obtain an Image that has the same 
number of rows and columns as the orignaly Input noisy Image, the first and last two rows and columns are copied 
directly to the output buffer, although they may be temporarily stored prior to insertion Into the output buffer. This step 
Is shown as the block 40 of Figure 5, the filter output block. 

[0043] Figures 13A and 13B show an original image and an image taken from the output of an embodiment of the 
adaptive noise filter 1 0, respectively. The adaptlveness of the algorithm is clearly visble by viewing these figures. For 
Instance, the walls and paper on the desk have a much reduced noise-value, but detail Is not lost around the buttons 
of the phone, for Instance. 

[0044] Figure 14 Is a diagram generated showing the strength of the filtering of Figure 1 3A to produce Figure 13B. 
The light grey zones represent heavily filtered areas, and dark grey zones represent lightly filtered areas. The paper 
on the desk had one of the strongest amounts of noise filtering applied to It, while the borders of the objects get almost 
no filtering at all — to preserve their details. One of the reasons for this is seen with reference to equation B above. 
That equation forces Kl, which Is directly Involved In the output pixel equation of equation 9 to zero when the distance 
Dl Is greater than the second threshold, Th2. At the borders of objects, there will be a large distance In pixel values, 
because of the sharp edge. Using the adaptive filtering techniques described herein, the sharp edges (and hence 
detail) is preserved, while other noise, such as In large blocks of monochrome color (as In sheets of paper, walls, and 
other large surfaces) gel a strong amount of noise filtering, thus producing an overall high quality image. 
[0045] In testing different embodiments of the processes, a synthetic image was used to determin how much noise 
the algorithm is capable of removing. Figure 15 is the test Image, which has an increasing amount of noise from top 
to bottom. To modulate the power of the added noise, the variance value of a gaussian distribution was manipulated. 
The mean value of the noise statistical distribution is assumed to be zero. 
[0046] If: 

'noisy denotes a noisy Bayer pattern; 

'filtered denotes the Bayer pattern filtered by the adaptive noise filter 1 0; and 
'original denotes the original noiseless Bayer pattern, 
[0047] Then: 

'noisy " 'original ~ Udded.noisd and 
'filtered ■ Original = 'residualjioise' 

[0048] Where Udded.noise ,s tne image that contains only the noise that was artificially, added to (original- Whereas 
Iresidual_noise 's the Image containing the residual noise after filtering. Given an Image x(n ( m), NX noise power 
can be computed according to the following formula: 



i 1V-I M l 



(10) 
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[0049] Noise power has been computed for both Udded.noise and 'residual^noise "sing the Image of Figure 15, 
transformed Into decibel <dB) form, and graphed In Figure 1 8. The y-axls of the graph In Figure 1 6 represents th e noise 
power expressed in dBs, and the x-axis represents the variance of the noise distribution. The difference between the 
two plots Indicates how much noise was reduced using the embodiment of the adaptive noise filter 1 0. in this test, the 
noise power of the filtered Image Is about 7 dB less than the original noise power, however, the adaptive noise filter is 
still able to discriminate between high texture and low texture areas. 

[0050] One of the attractive benefits of embodiments of the adaptive noise filter 10 is the ability to work directly on 
Bayer pattern data, it adaptations are color specific, and therefore produce better overall results than if they are simply 
averaged. Using gain factors from the autoexposure data combined with the human visual system, excellent pre- 
determinations are able to be made about how much filtering can be safely applied to specific pixels. Also, the noise 
filter works quickly because of Its ability to recursively compute based on pixel values already calculated, using less 
amount of memory In a hardware Implementation. 

[0051] Changes can be made to the invention In light of the above detailed description, in general, in the following 
claims, the terms used should not be construed to limit the Invention to the specific embodiments disclosed In the 
specification and the claims, but should be construed to Include all methods and devices that are in accordance with 
the clafcns. Accordingly, the Invention is not limited by the disclosure, but Instead its scope Is to be determined by the 
following claims. 



Claims 

1. Improved Image filter for a digital camera Including Image sensors sensitive to light, a color filter placed over 
sensitive elements of the sensors and patterned according to a Bayer mosaic pattern layout and an interpolation 
algorithm Joining togetherthe digital Information provided by differently colored adjacent pixels In said Bayer pat- 
tern, characterized In that said fitter (1 0) is adaptive and includes a noise level computation block (26) for operating 
directly on a said Bayer pattern data set of for each color channel thus removing noise while sfrnultaneously pre- 
serving picture detail. 

2. improved Image filter according to claim 1 wherein at least two filter masks are used, one for the green channel 
and the other forthe red and blue channels. 

3. Improved Image filter according to claJ m 2 wherein the Ulter masks for the red and blue channels are Identical while 
the filter mask for the green channel Is different. 

4. Improved image filter according to claim 1 1ncluding: a determiner block (1 6) for examining a current pixel (TO) In 
a processing window and determining If It Is a green pixel to select a corresponding filtering mask; a difference 
calculator (22) for determining an absolute value difference (Di) between said current pixel (TO) and the values of 
its neighboring pixels In said processing window; a maximum and minimum block (24) for computing the maximum 
and minimum distance values from a list of difference values (D1 D8); a noise level computation block (26) for 
estimating a noise level (NL) associated with said processing window. 

5. Improved Image filter according to claim 4 further including a gain factor block (34) comprising an autoexposure 
control system for generating an appropriate set of exposure and gain settings fed to said noise level computation 
block (26). 

6. Improved Image filter according to claim 4 wherein said noise level computation block (26) uses a recursive Im- 
plementation and extlmates the noise level according to the following calculations: 

NL R (t-1) = K n (M) * D max (t-1) + [1 - K n (M)J * NL R (t-2) (1) 



NLq CM) = K n (M) * O max (M) + [1 - K„ (M)] * NL G (t-2) (2) 



NL B (t-1) = K n (M) * D max (M) + [1 - Kn (t-1)J * NL B (t-2) 



(3) 



BP 1299 309 A1 



where NL* Is the noise level of the particular noise channel, x; K„ Is a parameter that determines the strength 
of filtering to be performed by the adaptive noise filter (10), and D mM Is the maximum distance value computed 
for the particular color channel In max/mln block (24). 

7. A digital camera, comprising: 

a sensor for sensing an Image and producing a first signal; 

a Bayer pattern producer coupled to said sensor and structured to produce a Bayer pattern from said first signal; 
a splitter structured to split said Bayer pattern Into separate color channels; 
characterized In that It further comprises an image filter according to claim 1. 

8. Method for filtering noise In a digital camera Including Image sensors sensitive to light, a color filter placed over 
sensitive elements of the sensors and patterned according to a Bayer mosaic pattern layout and an Interpolation 
algorithm joining together the digital Information provided by differently colored adjacent pixels in said Bayer pat- 
tern, characterized by the following steps: 

- scanning sequentially row by row the Bayer pattern format; 

- providing a processing window for each color channel of said Bayer pattern data set; the processing windows 
fortheredandbluechannels being Identical, while the processing window for the green channel being different; 
each processing window having a target pixel and a plurality of neighboring pixels; 

- selecting afilterlng mask and acorrespon^ 
pixel color Is green; 

- filtering the target pixel computing also the noise level (NL) of a color channel and providing a filtered pattern 
format depending on said noise level. 

9. Method according to claim 8, characterized In that the filtered value of the current target pixel (TO) is computed 
using the pixel values that have already been filtered during said scanning step. 
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